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ABSTRACT 

Office  automation  has  been  credited  as  a  means  by  which  office  and 
organizational  productivity  can  be  increased.  Incorporation  of  office 
automation  technology  can  represent  a  large  investment  for  any 
organization.  To  guarantee  the  proper  allocation  of  resources, 
government  managers  and  executives  must  utilize  evaluation  techniques 
which  insure  that  the  benefits  of  a  program  outweigh  the  costs.  Naval 
Laboratories  have  been  planning  to  purchase  office  automation 
technology  to  increase  their  productivity.  To  insure  that  the  benefits 
of  office  automation  can  be  realized  and  are  cost  effective  an 
evaluation  method  should  be  developed  which  quantifies  increases  in 
productivity.  The  purpose  of  this  thesis  is  to  present  an  evaluation 
method  which  provides  ostensive  and  confirmable  evidence  with  regard 
to  productivity  changes  and  the  cost /benefit  of  office  automation.  The 
Naval  Laboratories  Technical  Office  Automation  and  Communications 
System  (NALTOACS)  Pilot  Assessment  and  the  Automated  Technical 
Information  Processing  System  (ATIPS)  at  Naval  Weapons  Center  (NWC) 
China  Lake,  California  were  used  as  the  basis  of  analysis  for  the 
development  of  the  evaluation  method. 
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I.   INTRODUCTION 

The  goal  of  this  thesis  is  to  present  a  method  for  determining 
whether  office  automation  (OA)  increases  productivity  in  Naval 
Laboratories  and  whether  OA  is  cost  effective  for  those  laboratories. 
To  this  end  the  Naval  Laboratories  Technical  Office  Automation  and 
Communications  System  (NALTOACS)  Pilot  Assessment  and  the  office 
automation  pilot  system  at  Naval  Weapons  Center,  China  Lake, 
California  have  been  analyzed.  An  overview  of  the  major  issues  of  this 
thesis  is  contained  in  this  chapter. 

This  study  addresses  three  major  issues:  productivity,  the  costs 
and  benefits  (cost/benefit)  of  office  automation,  and  the  methodology 
used  to  determine  productivity  and  its  cost/benefit.  The  first  issue, 
productivity,  is  a  measure  of  efficiency  and  effectiveness  (Greenwood, 
1984).  Efficiency  involves  comparing  inputs  to  outputs:  how  much  or 
how  little  input  is  required  for  acceptable  output  or  conversely  how 
much  output  is  acceptable  given  a  fixed  level  of  inputs.  Effectiveness 
involves  attainment  of  goals:  for  example,  how  a  program  or  system 
affected  profits  if  profits  were  the  organizational  goal.  The  second 
issue,  cost/benefit,  is  addressed  by  comparing  the  overall  benefit 
attained  from  the  system  with  the  overall  cost.  This  is  accomplished 
after  a  measure  of  office  productivity  has  been  determined.  The  third 
issue,  the  methodology  used  to  determine  productivity  and  cost/ 
benefit,  is  addressed  through  analysis  of  the  methodology  used  by 


ccmmuni cations,  computerized  branch  exchanges,  and  data  base 
management  systems  (Levine,  1985). 

B.   POTENTIAL  BENEFITS  OF  OFFICE  AUTOMATION 

Computer  technology  was  initially  sought  by  industry  as  a  means  of 
cutting  secretarial  and  clerical  costs.  Office  employment  is  estimated 
to  be  30%  of  the  total  labor  force  (Panko,  1985).  Greenwood  (1984) 
estimated  that  35%  of  the  office  labor  force  is  involved  in 
secretarial  or  clerical  work.  Given  these  estimates,  secretarial/ 
clerical  work  is  estimated  to  comprise  approximately  10%  of  the  total 
labor  effort  in  the  workplace.  Office  automation  has  clearly  proven 
that  it  can  increase  secretarial  efficiency  through  the  use  of  word 
processing  equipment.  Law  and  Pereira  (1976)  have  shown  through 
exper •'.mentation  that  the  average  secretary  can  only  type  ten  words  per 
minute  when  errors  and  disturbances  were  factored  in.  In  their 
experiments,  word  processing  equipment  use  was  judged  responsible  for 
secretaries  increasing  their  output  to  thirty  words  per  minute:  a  300% 
increase  in  output.  Additionally,  Shiff  (1974)  proposed  that 
implementation  of  computerized  office  equipment  could  eventually 
eliminate  typing,  transcription,  and  filing  from  office  work:  thus 
eventually  eliminating  the  secretary,  too. 

Apart  from  the  reduction  in  secretarial  help,  Gottheimer  (1979) 
suggests  that  automation  of  the  office  would  significantly  reduce  the 
need  for  middle  managers.  This  can  be  done  by  making  executives  more 
efficient.  Macfarlane  (1983)  sights  several  aspects  of  OA  systems  that 
are  important  to  management  productivity.  Information  retrieval  can 
save  executives  time  by  allowing  them  to  access  personal  and  corporate 


information  from  their  desk.  Analytical  tools  such  as  spreadsheets  can 
be  used  to  analyze  information  once  it  has  been  retrieved.  Special 
applications  programs  such  as  graphics  allow  presentation  of 
information  in  a  manner  that  makes  reports  more  persuasive.  Electronic 
mail  allows  information  exchange  in  a  faster  and  more  informal  fashion 
than  normal  mail  which  leads  to  a  greater  exchange  of  ideas  and 
information. 

In  addition  to  increasing  efficiency,  OA  can  have  other  tangible 
and  intangible  benefits.  Williams  and  Jones  (1984)  list  the  following 
as  tangible  benefits: 

-  equipment  charges  eliminated 

-  temporary  help  not  needed 

-  overtime  reductions 

-  work  force  reductions 

They  also  list  the  following  as  intangible  benefits: 

-  improved  communications 

-  enhanced  document  appearance 

-  less  time  to  prepare  proposals  and  reports 

Additionally,  the  Honeywell  Technalysis  Office  Management  Systems 
Division  conducted  a  national  survey  in  1985  to  examine  the  attitudes 
of  office  workers  who  have  access  to  OA  equipment.  That  survey 
revealed  that  90%  of  those  surveyed  reported  that  due  to  OA  they  had 
more  time  to  devote  to  creative  activities.  Those  surveyed  also 
reported  that  OA  played  a  major  role  in  helping  them  make  better  and 
more  informed  decisions.  About  half  of  those  surveyed  stated  that  OA 
equipment  is  responsible  for  increases  in  speed,  efficiency,  and 
productivity. 

OA  has  been  identified  as  a  means  for  organizations  to  increase 
their  productivity  while  decreasing  their  costs  (Forest,  1979; 
Gottheimer,  1979;  Williams  and  Jones,  1984). 
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C.  IS  OFFICE  AUTOMATION  REALLY  BENEFICIAL? 

There  is  a  fear  throughout  industry  that  the  potential  benefits 
extolled  by  OA  champions  will  not  be  realized.  The  installation  of  new 
technology  per  se  is  not  the  way  to  make  offices  more  efficient  and 
productive,  according  to  Charles  Callahan  (EDP,  1985).  Callahan 
contends  that  office  automation  cannot  be  justified  by  the  vague  claim 
that  OA  will  "increase  productivity"  (p.  4);  "management  wants  bottom 
line  payoffs"  (p.  4).  Ralph  Kleim  (1985)  wrote  that  using  OA  systems 
does  not  necessarily  mean  a  bona  fide  increase  in  productivity. 

Kathleen  Foley  Curley  (1984)  summerizes  the  skepticism  about 

office  automation. 

A  real  fear,  particularly  among  line  managers,  is  that  new  office 
automation  technologies  "may  make  us  the  best  informed,  unprofitable 
company  in  our  market."  How  much  information  is  too  much?  What  are 
the  benefits  to  be  gained  and  how  can  they  be  quantified?  There  is 
also  a  question  of  whether  an  investment  in  office  automation 
technology  is  appropriate  in  every  business  or  in  every  industry, 
(p.  37) 

The  notion  that  OA  may  not  be  appropriate  for  every  business  or  in 

every  industry  leads  to  the  following  question:  how  can  a  business  or 

industry  determine  whether  OA  is  appropriate  for  their  needs? 

Specifically  for  this  thesis,  how  can  Naval  Laboratories  determine 

whether  possible  OA  benefits  will  be  realized? 

D.  OFFICE  AUTOMATION  AND  NAVAL  LABORATORIES 

The  Navy  currently  has  eight  laboratories  that  perform  a  wide 
variety  of  research  functions.  The  goal  of  these  laboratories  is  the 
development  of  new  or  improved  equipment  and/or  techniques  to  support 
of  the  missions  of  the  Department  of  Defense. 

As  early  as  1979  then  Secretary  of  the  Navy  (SecNav),  W.  Graham 
Claytor,  Jr. ,  stated  in  a  memorandum  to  the  Chief  of  Naval  Operations 
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that  administrative  modernization  in  the  Department  of  the  Navy  was 
required  to  oope  with  "the  volume  of  correspondence  and  administrative 
material"  (SecNav,  July  1979).  In  1982  the  Naval  Laboratories 
Technical  Office  Automation  and  Communications  System  (NALTOACS) 
program  was  established  to  coordinate  the  introduction  of  office 
systans  technology  into  Naval  laboratories  as  a  means  of  enhancing 
productivity  (NALTOACS,  1983).  Previous  to  the  establishment  of 
NALTOACS,  three  of  the  eight  Naval  Laboratories  had  introduced  OA 
technology.  They  are: 


-  David  W.  Taylor  Naval  Ship  Research  and  Development  Center 
(NSRDC),  Bethesda,  Maryland 

-  Naval  Surface  Weapons  Center  (NSWC) 
Dahlgren,  Virginia 

-  Naval  Weapons  Center  (NWC) 
China  Lake,  California 


David  W.  Taylor  NSRDC  was  assigned  as  the  lead  laboratory  and  was 
given  the  responsibility  for  NALTOACS.  The  Deputy  Under  Secretary  of 
the  Navy  for  Financial  Management  reaffirmed  a  requirement  from  the 
Commander,  Naval  Data  Automation  Command  that  an  assessment  of  the  OA 
pilot  projects  already  in  place  should  be  performed  prior  to  the 
initiation  of  any  further  OA  implementation  in  Navy  Laboratories.  The 
pilot  assessment  done  by  David  W.  Taylor  NSRDC  was  completed  without 
gathering  any  quantitative  evidence  (i.e.,  reduction  in  personnel 
costs  or  travel  costs)  relating  to  productivity  in  an  automated 
office.  The  study  drew  inferences  of  productivity  changes  by  gathering 
subjective  information  (i.e.,  user  opinions  and  estimates  of  how  much 
time  per  day  the  OA  system  saved  them)  from  laboratory  OA  users.  The 
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Naval  Laboratory  personnel  were  surveyed  and  asked  to  estimate  their 
perceptions  of  productivity  gains  made  possible  through  the 
installation  of  the  OA  system.  The  results  of  the  pilot  assessment 
done  by  David  W.  Taylor  NSRDC  were  published  in  the  "Pilot  Assessment 
Final  Report"  dated  April  29,  1983. 

E.   OFFICE  AUTOMATION  AT  NAVAL  WEAPONS  CENTER  (NWC) ,  CHINA  LAKE 

NWC,  China  Lake  is  the  principle  Navy  research,  development,  test 
and  evaluation  center  for  the  air  warfare,  electronic  warfare,  missile 
systems,  defense  suppression,  and  parachute  systems.  The  need  for 
office  automation  technology  at  NWC  China  Lake  was  documented  first  in 
Proposal  for  Establishment  of  an  Automated  Technical  Information 
Processing  System  (ATIPS)  (NWC,  July  1980).  This  proposal  sighted 
"heavily  increasing  paper  workloads  along  with  reductions  in  the 
manpower  currently  handling  those  workloads"  as  a  major  problem  at 
NWC.  Additionally,  delays  in  document  production  and  problems  with 
copying  and  storage  of  "phenomenal  amounts  of  paper"  contribute  to  the 
waste  of  "hundreds  of  thousands  of  dollars  each  year"  (NWC,  July 
1980). 

To  handle  this  problem  two  courses  of  action  were  recommended: 
"(1)  make  each  person  at  NWC  capable  of  doing  more  work  with  less 
effort  in  shorter  periods  of  time  and  (2)  reduce  dependence  upon  paper 
as  a  medium  of  cammunication"  (NWC,  July  1980,  p.  2). 

The  ATIPS  Concept  Development^ Paper  (NWC,  March  1982)  delineated 
the  functions  necessary  of  the  ATIPS  system  to  resolve  the  problems 
brought  forth  in  the  Proposal  for  Establishment  paper  (NWC,  July 
1980).  To  be  effective  ATIPS  would  need  to  provide  the  following: 
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1.  Text  Processing:  word  processing,  text  entry,  document 
editing  and  revision,  special  forms/reports  production. 

2.  Records  Management:  filing,  indexing,  and  retrieval  of 
information  on  request. 

3.  Information  Processing:  analytical  models  for  analysis/ 
computations/projections,  file  manipulation,  text/data 
merging,  business  graphics. 

4.  Management  Information:  allow  access  to  internal  and 
external  data  sources. 

5.  Communications:  provide  inter-office  connection  for 
information  exchange. 

6.  Personal  Services:  office  calendar,  daily  schedule, 
office  directory,  personal  records  handling  capability. 

Given  equipment  that  could  fulfill  the  functional  requirements 

listed  above  the  ATIPS  system  was  expected  to  provide  the  following 

benefits: 

1.  Reduce  administrative  workload  for  technical  and  administrative 
personnel . 

2.  Provide  ready  access  to  large  quantities  of  information. 

3.  Reduce  the  space  required  for  storage  of  information. 

4.  Reduce  administrative  costs. 

5.  Reduce  the  number  and  size  of  meetings. 

6.  Reduce  the  overall  labor  cost  for  each  productive  unit  of  work. 

7.  Make  NWC  a  more  cost  effective  Research,  Development,  Test  and 
Evaluation  (RDT&E)  center. 

The  foundation  of  the  ATIPS  system  is  the  Xerox  8000  STAR  system. 

This  system  was  in  its  developmental  stage  when  it  was  delivered  to 

NWC.  The  software  installation  was  provided  with  the  Xerox  system.  The 

telecommunications  equipment  was  provided  by  Xerox,  too;  the 

telecommunications  equipment  was  also  still  in  the  developmental 

stage.  Thus,  NWC  acted  as  a  test  site  for  the  Xerox  Ethernet 
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(NALTOACS,  April  1983).  The  ATIPS  system  was  initially  operational  in 
February  1982.  The  system  was  scheduled  to  grow  to  approximately  1500 
terminals  to  be  used  by  2500  people;  the  total  cost  of  the  system  was 
budgeted  at  $11,875  million  (NWC,  August  1981).  However,  ATIPS  funding 
was  not  continued  beyond  the  installation  of  the  pilot  system. 

F.   SUMMARY 

Evidence  indicates  that  there  are  potential  benefits  that  can  be 
realized  by  organizations  through  the  implementation  of  office 
automation  systems.  The  primary  benefit  claimed  by  OA  is  that 
productivity  will  increase  while  cost,  in  the  long  term,  will  be 
reduced.  However,  a  survey  of  current  literature  indicates  that  there 
is  skepticism  over  the  actual  realization  of  OA  benefits.  The 
Department  of  the  Navy  has  established  an  organization  (NALTOACS)  to 
supervise  the  implementation  of  OA  systems  in  Navy  Laboratories. 
However,  prior  to  the  implementation  of  OA  systems  in  Naval 
Laboratories  evidence  should  be  presented  that  specifically  supports 
the  claims  that  OA  increases  productivity  in  the  Naval  Laboratory.  The 
Pilot  Assessment  done  by  NALTOACS  included  information  on  the 
productivity  changes  experienced  by  the  three  pilot  programs  surveyed. 
However,  this  information  was  gathered  in  a  qualitative  manner.  Naval 
laboratory  personnel  were  asked  to  estimate  their  perceptions  of 
productivity  gains  made  possible  through  the  implementation  of  OA 
systems . 
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II.   NALTOACS  PILOT  ASSESSMENT  REVIEW 

Chapter  I  gives  an  overview  of  office  automation,  its  potential 
benefits,  the  current  status  of  OA  in  Naval  Laboratories,  and  the 
status  of  the  office  automation  system  at  NWC  China  Lake.  A  brief 
overview  of  the  Pilot  Assessment  of  the  three  Naval  Laboratory  OA 
pilot  projects,  done  by  NALTOACS,  is  also  presented. 

This  chapter  analyzes  the  NALTOACS  Pilot  Assessment  in  detail. 
This  analysis  is  not  concerned  with  the  results  of  Pilot  Assessment 
but  rather  with  the  method  by  which  NALTOACS  performed  their 
assessment.  To  perform  an  analysis  of  the  NALTOACS  evaluation  method 
it  is  necessary  to  begin  by  addressing  evaluations  in  a  very  general 
sense:  what  they  are,  what  their  purpose  is,  and  why  they  are  useful. 
Next,  the  two  main  types  of  evaluations,  quantitative  and  qualitative, 
are  explained  including  their  strengths  and  weaknesses.  The  issue  of 
evaluation  objectivity  vs.  subjectivity  is  also  discussed. 
Additionally,  this  chapter  explains  why  the  NALTOACS  Pilot  Assessment 
is  an  inadequate  evaluation  of  the  impact  of  office  automation 
technology  in  Naval  Laboratories. 

A.   EVALUATION  DEFINITION 

What  is  an  evaluation?  Deming  defines  evaluation  as  a 
pronouncement  concerning  the  effectiveness  of  some  treatment  or  plan 
that  has  been  put  into  effect  (Guttentag  and  Struening,  1975).  It  is  a 
"process  used  to  analyze  the  relationship  between  actual  and  desired 
effects"  (Euske,  1984,  p.  53).  Evaluation  is  the  measurement  of 
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performance  and  the  making  of  comparisons  based  on  those  measurements 
(Wholey,  1979).  "Evaluation  is  the  measurement  of  desirable  and 
undesirable  consequences  of  an  action  intended  to  forward  some  goal 
that  the  actor  values"  (Riecken,  1953,  p.  1).  All  of  these  definitions 
have  measurement  and  comparison  as  common  ground.  An  evaluation  may  be 
searching  for  the  effector  of  performance  or  an  evaluation  may  be 
searching  for  the  effect  that  an  action  has  on  performance;  in  both 
situations  there  is  commonality  in  necessity  for  measurement  and 
comparison.  For  the  purpose  of  this  thesis  Riecken 's  definition  of 
evaluation  will  be  used. 

Riecken 's  definition  of  evaluation  is  very  broad  and  general.  This 
thesis  is  concerned  with  "action"  defined  as  the  installation  of  an 
office  automation  system.  In  the  following  sections  of  this  chapter 
this  definition  is  further  refined. 

B.   PURPOSE  OF  EVALUATION 

Anderson  and  Ball  (1978)  list  six  major  uses  and  needs  for 
evaluation: 

1.  To  contribute  to  decisions  about  program  installation. 

2.  To  contribute  to  decisions  about  program  continuation, 
expansion,  or  certification. 

3 .  To  contribute  to  decisions  about  program  modification. 

4.  To  obtain  evidence  to  rally  support  for  a  program. 

5.  To  obtain  evidence  to  rally  opposition  to  a  program. 

6.  To  contribute  to  the  understanding  of  basic  psychological, 
social,  and  other  processes,  (pp.  3-4) 

This  thesis  is  most  concerned  with  purposes  1,  3,  and  4.  These  three 

purposes  are  all  basically  the  same  in  substance  but  with  shades  of 
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difference  in  timing  and  user/evaluator  orientation  or  bias.  The  issue 
of  timing  involves  whether  a  program  is  in  place  or  not;  purpose  one 
is  oriented  toward  a  new  program  that  is  not  yet  in  place  while 
purpose  three  and  four  are  oriented  toward  a  program  that  is_  already 
in  place.  The  issue  of  user/evaluator  orientation  or  bias  involves  the 
objectivity  of  the  user/evaluator;  purpose  one  is  concerned  with 
gathering  evidence  in  an  objective  manner  while  purpose  three  and  four 
are  looking  for  evidence  in  a  positive  or  negative  sense.  Funda- 
mentally, all  three  of  these  purposes  can  be  condensed  into  a  single 
concept:  the  purpose  of  an  evaluation  is  to  obtain  evidence  about  the 
effectiveness  of  a  program. 

The  purpose  of  evaluation  that  I  propose  above  is  very  broad  and 
general.  From  this  point  forward  this  thesis  will  narrow  the  term 
"program"  used  in  the  explanation  of  evaluation  purpose  to  "the 
installation  of  office  automation  technology."  Therefore,  for  this 
thesis  the  purpose  of  an  evaluation  is  to  obtain  evidence  about  the 
effectiveness  of  the  installation  of  office  automation  technology. 

C.   USEFULNESS  OF  EVALUATIONS 

In  general,  evaluations  are  decision  tools;  they  are  an  aid  in 

making  choices  (House,  1980).  With  regard  to  the  issue  of 

incorporating  office  automation  technology  at  Naval  Laboratories,  a 

government  manager,  executive,  or  official  could  use  an  evaluation  as 

a  aid  in  making  choices  about  providing  office  automation  equipment  to 

Naval  Laboratories.  However,  evaluations  have  a  special  usefulness  to 

government  employees  as  written  by  Franklin  and  Thrasher  (1976): 

In  an  era  of  heightened  public  concern  about  the  distribution  and 
management  of  public  funds,  at  a  time  of  diminished  buying  power, 
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when  ideology  of  a  "good  return  on  an  investment"  competes  with  more 
charitable  and  affluent  ideologies  for  ascendancy  in  the  public 
ethos,  evaluation  becomes  a  prominent  and  visible  concern  for  the 
managers  of  public  programs,  (p.  2) 

In  a  democracy  ultimate  accountability  is  to  the  public;  the  public, 
by  direct  expression  and  through  its  elected  representatives,  is 
increasingly  demanding  a  rendering  of  that  account,  (p.  3) 

According  to  Joseph  Who ley  (1979),  those  in  charge  of  government 
programs  must  make  a  countless  number  of  decisions  under  tremendous 
time  constraints,  often  with  little  reliable  information  on  program 
performance.  He  writes  that  the  key  to  better  decisions  and  better 
government  programs  is  the  establishment  of  realistic  and  measurable 
objectives  and  measures  of  program  performance  and  the  use  of  program 
performance  information  to  bring  about  changes  in  program  activities 
that  will  ultimately  enhance  program  performance.  In  short,  Who ley 
advocates  using  program  evaluation  as  the  major  decision  aid  for 
government  managers. 

In  summary,  an  evaluation  of  office  automation  systems  at  Naval 
Laboratories  is  the  measurement  of  desirable  and  undesirable 
consequences  due  to  OA  installation.  The  purpose  of  an  evaluation  of 
Naval  Laboratory  OA  systems  is  to  obtain  evidence  about  their 
effectiveness.  This  evidence  is  useful  to  government  managers  and 
executives  who  are  responsible  for  Naval  Laboratory  Office  Automation 
Programs  because  evaluations  enhance  program  performance  in  an  era 
when  maximum  efficiency  and  effectiveness  is  demanded  by  the  public. 

D.   TYPES  OF  EVALUATION 

Michael  Quinn  Patton  (1978)  writes  that  there  are  two  main  types 
of  evaluation:  quantitative  and  qualitative.  He  explains  that  these 
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two  types  of  evaluation  are  due  to  the  two  basic  paradigms  that 
dominate  evaluative  techniques.  The  first  paradigm,  which  he  says  is 
the  dominant  paradigm  in  evaluation,  espouses  the  hypothetico- 
deductive  methodology.  This  is  more  commonly  known  as  the  scientific 
method;  it  assumes  that  quantitative  measurement,  experimental  design, 
and  statistical  analysis  is  the  epitome  of  good  science  (Patton, 
1978).  The  second  paradigm,  which  Patton  calls  the  alternative 
paradigm,  espouses  the  holistic- inductive  methodology.  This  paradigm 
relies  on  qualitative  data,  holistic  analysis,  and  detailed 
description  derived  from  close  contact  with  the  targets  of  study 
(Patton,  1979). 

In  what  situation  is  it  appropriate  to  use  each  of  these  methods? 
Patton  (1979)  writes  that  the  hypothetico-deductive  (quantitative) 
approach  aims  at  performance  and  prediction  while  the  holistic- 
inductive  (qualitative)  approach  aims  at  understanding.  Kuhn  wrote: 
"the  most  deeply  held  values  concern  predictions:  they  should  be 
accurate;  quantitative  predictions  are  preferable  to  qualitative  ones" 
(Patton,  1978,  p.  184).  Patton  (1978)  contends  that  for  predictions: 
"qualitative  analyses  in  general  have  little  legitimacy  beyond  certain 
limited  exploratory  situations"  (p.  211).  However,  Patton  (1978)  does 
concede  that  there  are  situations  where  quantitative  measurement  is 
impossible  (as  in  cases  where  an  attempt  to  measure  "goodness"  is 
made)  and  where  the  qualitative  method  is  the  only  practical  method. 
He  also  states  that  to  attain  the  goals  set  for  some  evaluations  it 
may  be  necessary  to  use  the  qualitative  method. 

Analogous  to  the  two  types  of  evaluation,  quantitative  and 
qualitative,  is  evaluation  objectivity  and  subjectivity.  Evaluation 
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objectivity  is  the  ability  to  evaluate  without  the  influence  of 

personal  feelings  (House,  1980).  Evaluation  subjectivity  is  evaluation 

by  personal  feelings.  Objective  measures  are  distinguished  from 

subjective  measures  by  the  presence  or  absence  of  ostensive  indicators 

(Guttentag  and  Struening,  1972).  Ostensive  indicators  are  hard  data: 

data  that  is  external  to  an  evaluator/observer,  data  that  can  be 

specified  and  measured  using  some  agreed  upon  measurement  tools. 

Patton  (1978)  writes: 

Qualitative  methodology  and  a  responsive,  illuminative  approach  to 
evaluation  most  frequently  stimulate  charges  of  subjectivity  -  a 
label  regarded  as  the  very  antithesis  of  scientific  inquiry. 
Objectivity  is  considered  the  sine  qua  non  of  the  scientific  method. 
To  be  subjective  means  to  be  biased,  unreliable,  and  nonrational. 
Subjective  data  implies  opinion  rather  than  fact,  intuition  rather 
than  logic,  impression  rather  than  confirmation.  Scientists  are  to 
eschew  subjectivity  and  make  their  work  "objective  and  value  free", 
(p.  216) 

This  quotation  from  Patton  should  not  be  construed  that  opinion, 

intuition,  and  impression  are  mutually  exclusive  of  fact  and  logic. 

The  key  point  is  that  opinion,  intuition,  and  impression  are  not 

conf irmable.  Evaluations  should  seek  evidence  that  is  ostensive  and 

conf irmable. 

Which  of  these  two  methodologies  should  be  chosen  to  evaluate  a 

program?  Patton  (1978)  wrote  that  different  kinds  of  problems  require 

different  types  of  research  methodology.  NALTOACS  "was  established  to 

coordinate  the  introduction  of  office  systems  technology  into  the  Naval 

Laboratory  as  a  means  to  enhance  productivity."  (NALTOACS,  1983,  p.  1) 

The  objectives  for  the  NALTOACS  Pilot  Assessment,  were  to  gather 

information  relating  to  OA  system  planning,  implementation,  and 

benefits  analysis  (NALTOACS,  1983).  The  first  two  objectives, 
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gathering  planning  and  implementation  information,  are  questions  of 
understanding;  these  objectives  can  best  be  attained  through  use  of 
the  qualitative  methodology  (Patton,  1979).  The  last  objective,  the 
benefits  analysis,  is  a  question  of  predicting  performance.  This  is 
the  most  important  objective  of  the  Pilot  Assessment  because  it 
determines  whether  the  project  is  worthwhile:  no  matter  how  well 
planned  or  implemented  a  system  is  the  bottom  line  is  cost/benefit;  if 
the  benefits  do  not  exceed  the  costs  then  the  system  is  not  worth 
planning  or  implementing.  To  perform  a  cost/benefit  analysis  the 
quantitative  methodology  should  be  employed;  it  is  the  method  that 
results  in  a  true  bottom  line  of  dollars  and  cents  and  it  is  the 
method  that  provides  confirmable  results. 

The  Pilot  Assessment  of  the  Naval  Laboratories  that  had  pilot 
office  automation  systems  in  place  was  an  evaluation  using  the 
qualitative  methodology.  Appendix  A  contains  a  sample  of  the 
questionnaires  and  interview  sheets  used  by  NALTOACS  to  perform  their 
evaluation.  These  questionnaires  clearly  sought  the  opinions  and  the 
perceptions  of  the  system's  users  and  managers.  This  thesis  does  not 
take  issue  with  the  selection  of  this  methodology  to  gather 
information  about  OA  system  planning  and  implementation.  However,  this 
methodology  is  inadequate  to  perform  a  cost/benefit  analysis  because 
it  does  not  return  ostensive  data.  Also,  there  are  biases  inherent  in 
the  tools  of  the  qualitative  methodology,  questionnaires  and 
interviews,  that  cause  the  results  evaluations  using  this  methodology 
to  be  suspect.  These  biases  are  discussed  in  the  next  section. 
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E.   QUALITATIVE  METHODOLOGY  SOURCES  OF  INVALIDITY  IN  THE  NALTOACS 
PILOT  ASSESSMENT 

To  this  point  information  has  been  presented  about  the  identity, 
purpose,  and  usefulness  of  evaluations  in  general.  Also,  the  two  basic 
types  of  evaluations  have  been  presented  with  their  perspective 
strengths  and  weaknesses  with  an  argument  why  the  quantitative 
methodology  is  preferable  to  the  qualitative  methodology  for  the 
performance  of  a  benefits  analysis.   It  has  been  established  that  the 
Pilot  Assessment  conducted  by  NALTOACS  used  the  qualitative 
methodology:  the  questionnaires  and  personal  interviews  are  subjective 
in  nature  (see  Appendix  A);  this  method  invited  opinion,  intuition, 
and  impression  which  is  difficult  if  not  impossible  to  confirm  as 
fact.  This  section  addresses  the  biases  and  sources  of  invalidity  that 
result  from  surveys  and  questionnaires  themselves.  The  NALTOACS  Pilot 
Assessment  is  sited  to  exemplify  the  biases  that  explained  in  this 
section. 

Webb  et  al.  (1966)  list  three  broad  sources  of  bias  and 
invalidity  when  the  qualitative  methodology  is  used: 

1.  errors  that  may  be  traced  to  those  being  studied, 

2.  errors  that  come  from  the  investigator, 

3.  errors  associated  with  sampling  imperfections. 

It  is  beyond  the  scope  of  this  thesis  to  address  the  latter  two 

sources  of  evaluation  invalidity.  However,  errors  that  may  be  traced 

to  those  being  studied,  called  the  reactive  measurement  effect  by  Webb 

et  al.  (1966),  are  discussed  in  detail. 

The  most  understated  risk  to  valid  interpretation  is  the  error  pro- 
duced by  the  respondent.  Even  when  he  is  well  intentioned  and  coop- 
erative, the  research  subject's  knowledge  that  he  is  participating 
in  a  scholarly  search  may  confound  the  investigator's  data.  Four 

23 


classes  of  this  type  of  error  are:  awareness  of  being  tested,  role 
selection,  measurement  as  a  change  agent,  and  response  sets.  (p.  13) 

Of  the  four  of  reactive  measurement  effect  errors  listed  above,  only 

awareness  of  being  tested,  role  selection,  and  response  sets  are 

applicable  to  this  thesis;  they  are  discussed  below. 

1.  The  Awareness  of  Being  Tested 

The  awareness  of  being  tested  has  been  called  the  "reactive 

effect  of  measurement,,  bias  by  Campbell  (1957).  Selltiz  et  al.  (1959) 

wrote: 

If  people  feel  that  they  are  "guinea  pigs"  being  experimented  with, 
or  if  they  are  being  "tested"  and  must  make  a  good  impression,  or  if 
the  method  of  data  collection  suggests  responses  or  stimulates  an 
interest  the  subject  did  not  previously  feel,  the  measuring  process 
may  distort  the  experimental  results,  (p.  97) 

This  does  not  mean  that  all  subjects  that  are  aware  that  they  are 

part  of  a  test  will  give  inaccurate  responses.  However,  the 

probability  of  bias  is  high  in  any  study  in  which  a  respondent  is 

aware  that  he  is  the  subject  of  a  test  (Webb  et  al . ,  1976). 

The  subjects  in  the  NALTOACS  Pilot  Assessment  were  definitely 

aware  that  they  were  part  of  an  experiment.  The  "NALTOACS  PILOT 

ASSESSMENT  USER  INTERVIEW  CHECKLIST  PACKAGE"  (see  Appendix  A)  contains 

the  following  instructions: 

The  data  being  collected  by  this  survey  is 
required  to  conduct  the  "Pilot  Assessment" 
directed  by  the  Deputy  Under  Secretary  of  the 
Navy  (Financial  Management)  and  the  Commander, 
Naval  Data  Automation  Command. 

Please  take  this  opportunity  to  candidly  express 
your  opinions  and  needs  so  that  the  overall 
planning  for  office  automation  (OA)  in  the  Naval 
Laboratories  can  directly  benefit  from  them.  Also, 
please  feel  free  to  add  any  inputs  which  you  may 
have  and  which  are  not  covered  by  the  checklist. 


24 


These  instructions  could  influence  the  subject  to  give  responses  so 
that  they  make  a  "good  iirpression"  as  described  above  by  Selltiz. 
There  are  at  least  two  reasons  for  this.  First,  the  survey  has  been 
directed  by  the  Deputy  Under  Secretary  of  the  Navy  who  is  in  authority 
far  above  the  respondents  in  the  chain  of  command.  This,  coupled  with 
the  questionnaire  requirement  for  respondents  to  sign  the  survey  may 
have  resulted  in  an  unwillingness  by  the  respondents  to  give  responses 
that  would  "create  waves".  The  second  reason  is  related  to  the  first 
in  that  the  collector  of  the  survey  is  from  Washington  and  is,  in  some 
sense,  in  authority  over  the  installation  where  the  respondents  work 
and  the  OA  equipment  is  installed.  There  might  be  a  tendency  to  give 
positive  responses  to  avoid  "telling  on  oneself"  as  an  organization 
and  incurring  scrutiny  from  above. 
2.  Role  Selection 

Role  selection  is  another  way  that  a  respondents  awareness  of 
research  may  produce  bias.  This  bias  may  not  be  a  defensive  reaction 
or  dishonesty  but  rather  a  specialized  selection  of  a  "proper 
behavior"  by  the  respondent  (Webb  et  al.,  1976).  The  contention  by 
Webb  et  al .  here  is  that  a  person  who  is  singled  out  by  an 
experimenter  forces  the  subject  into  a  role  defining  decision,  a 
decision  as  to  what  type  of  person  should  the  respondent  be  to  answer 
the  questions. 

The  "NALTOACS  PILOT  ASSESSMENT  USER  CHECKLIST  PACKAGE"  (see 
Appendix  A)  gives  the  respondent  an  opportunity  to  select  a  role  even 
before  the  respondent  takes  the  survey.  After  the  instructions  a  "user 
profile"  section  asks  the  respondent  for  the  following  information: 
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USER  PROFILE 

Date: 

Name: 

Code: 

Telephone: 

Bldg: 

How  would  you  classify 

your 

job  responsibilities: 

Command  and  Support 
Executive 
Manager 
Supervisor 
Professional 
Clerical 
Other 

Technical 

Manager 

Scientist 

Systems  Engineer 

Programmer/ Anal ys t 
"  Other 

The  implication  of  this  is  that  the  respondents  could  "play  a  role", 
as  suggested  by  Webb  et  al . ,  and  answer  questions  not  as  they 
personally  feel  but  as  they  feel  an  executive,  a  manager,  or  a 
professional  should  answer  the  questions. 
3.  Response  Sets 

Response  sets  deal  with  the  way  that  questions  are  asked.  For 
example,  Sletto's  experimentation  led  to  his  assertion  that 
respondents  will  endorse  a  statement  more  frequently  than  they  will 
disagree  with  its  opposite  (1937).  Campbell  wrote  that  the  direction 
of  wording  can  definitely  influence  the  respondents  in  questionnaires 
and  interviews  (1965).  Webb  et  al .  danonstrated  that  respondents 
generally  have  a  preference  for  strong  statements  rather  than  moderate 
or  indecisive  statements  (1966).  They  also  demonstrated  that  sequences 
of  questions  asked  in  a  very  similar  format  produced  stereotyped 
answers,  such  as  a  tendency  to  select  a  righthand  or  lefthand 
response.  Further,  they  proposed  that  decreasing  attention  produces 
biases  from  the  order  of  presentation. 

The  NALTOACS  PILOT  ASSESSMENT  USER  INTERVIEW  CHECKLIST  PACKAGE 
(see  Appendix  A)  has  questions  that  could  cause  response  set  errors. 
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There  are  questions  that  ask  the  respondent  to  choose  an  answer  from 

among  a  list  of  applicable  answers.  However,  74%  of  the  first  answers 

listed  were  responses  that  were  positive  for  office  automation  in 

general  (see  Appendix  A,  p.  68).  For  example: 

10.  Please  check  the  appropriate  responses: 

CALENDAR 

Essential     Available  Use  frequently 

Nice  to  have   Not  available  Use  occationally 

Do  not  need    Do  not  know  Do  not  use 

if  available 

I  have   /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs     Easy  to  use 

Must  be  improved,        Difficult  to  use, 

how  so  how  so 


"Essential",  "Available",  "Use  frequently",  "Adequate  to  my  needs", 

and  "Easy  to  use"  all  possess  a  positive  connotation  while  "Do  not 

need",  "Do  not  know  if  available",  "Do  not  use",  "Must  be  improved" , 

and  "Difficult  to  use"  all  possess  a  negative  connotation.  There  are 

five  pages  of  questions  similar  to  the  one  above  covering  the  various 

capabilities  of  the  pilot  office  automation  systems. 

11.  How  do  you  rate  the  overall  responsiveness  of  the 
system  in  operation? 

Fast  (under  2  sec)        Slow  (5-10  sec) 


Adequate  (2-4  sec)        Too  slow  (10  sec  +] 


17.  In  what  way  has  the  system  assisted  you  in  the 
preparation  of  products? 

Get  the  job  done  with  less  effort,  how  much  so?  _% 

Better  quality  of  end  products,  how  so? 

Fewer  typos 

Better  appearance 

More  timely 

Other  (specify)  
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Seventy  percent  of  the  questionnaire  consisted  of  questions 
that  were  also  asked  in  a  similar  format.  The  questionnaire  was  twenty 
pages  long;  fourteen  of  these  pages  (see  Appendix  A,  questions  10  and 
18)  were  similar  in  format.  Due  to  the  length  and  repetition  of  the 
question  format  it  is  possible  that  stereotyped  answers  were  given  as 
Webb  et  al.  proposed  above.  The  sheer  length  of  this  questionnaire 
brings  into  play  the  Webb  et  al .  assertion  that  decreasing  attention 
can  produce  biases . 

F.   SUMMARY 

This  chapter  has  examined  the  NALTOACS  Pilot  Assessment 
qualitative  evaluation  methodology.  This  methodology  was  determined  to 
be  appropriate,  in  principal,  to  fulfill  the  first  two  objectives 
(gathering  information  about  OA  system  planning  and  implementation)  of 
the  NALTOACS  Pilot  Assessment.  However,  this  methodology  was 
considered  inadequate  to  fulfill  the  most  important  objective,  a 
cost/benefits  analysis.  There  are  two  reasons  why  the  qualitative 
methodology  is  inadequate  for  a  cost/benefits  analysis.  First,  results 
are  based  on  opinions  which  are  not  ostensive  and  confirmable  in 
nature.  For  example,  the  NALTOACS  Pilot  Assessment  made  quantitative 
estimates  of  productivity  gains  due  to  OA  incorporation  without 
performing  any  quantitative  analysis;  inferences  were  drawn  from 
questionnaires  and  interviews  -  inferences  of  fact  based  on 
respondents'  opinions.  Specifically,  the  Pilot  Assessment  (NALTOACS 
1983)  estimates  a  productivity  gain  of  8  to  12%  and  a  savings  per  year 
of  1,681  to  2,415  man-years  across  the  seven  Naval  Laboratories  (p. 
49).  This  thesis  does  not  contend  the  figures  that  NALTOACS  presents 
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as  inaccurate  per  se.  However,  there  is  no  ostensive  data  here  nor  a 
method  of  confirmation.  Thus,  the  conclusions  drawn  are  suspect. 

The  second  reason  that  the  qualitative  methodology  can  result  in 
erroneous  evaluation  conclusions  is  that  the  tools  of  this  methodology 
(questionnaires  and  interviews)  can,  in  and  of  themselves,  bias  the 
respondents.  Thus,  even  if  the  methodological  approach  was  appropriate 
to  perform  a  benefits  analysis  the  results  would  still  be  suspect  due 
to  these  errors  and  biases. 
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III.   PROPOSED  OA  BENEFITS  ANALYSIS  METHOD 

Chapter  II  critically  analyzes  the  NALTOACS  Pilot  Assessment 
method.  It  begins  with  a  discussion  of  evaluations  in  general, 
explains  the  strengths  and  weaknesses  of  the  two  main  types  of 
evaluation  methodologies,  and  discusses  evaluation  objectivity  and 
subjectivity.  Then  the  NALTOACS  Pilot  Assessment  methodology  is 
examined;  its  inadequacy  to  attain  its  most  important  objective  (an  OA 
benefits  analysis)  is  specifically  addressed. 

This  chapter  proposes  a  detailed  method  through  which  an  OA 
benefits  analysis  can  be  performed.  This  is  done  by  studying  the  ATIPS 
program  located  at  Naval  Weapons  Center  China  Lake,  California. 

As  previously  stated  in  Chapter  II,  the  corner  stone  of  the 
quantitative  methodology  is  the  scientific  method.  The  scientific 
method  dictates  that  quantitative  measurement  and  comparative  analysis 
is  the  essence  of  reliable  and  valid  evaluation  (Patton,  1978). 
Measurement  is  the  key,  without  it  there  can  be  no  comparative 
analysis.  The  first  task  in  tailoring  the  quantitative  evaluation 
method  into  a  specific  application  must  be  to  select  a  measurement 
method  that  is  consistent  with  the  objectives  of  the  evaluation.  The 
evaluation  objective  that  is  the  focus  of  this  chapter  is  an  OA 
cost /benefit  analysis.  Therefore,  a  method  to  measure  the  costs  and 
benefits  of  an  OA  system  must  be  devised. 

Prior  to  the  installation  of  any  program  or  system  there  should  be 
perceived  need  for  that  program/system.  This  will  be  the  starting 
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point  for  the  OA  system  performance  measurement  criteria.  Wholey  wrote 

that  an  evaluation  of  any  program  or  system  must  document  the  extent 

to  which  management  has  defined  the  measures  of  resource  inputs, 

program  activities,  outcomes,  and  impacts  (1979). 

There  are  two  parts  to  the  question  of  measurability:  (1)  the 
indicator  of  achievement  and  (2)  the  means  of  verification.  It  is  a 
standard  part  of  program  evaluation  methodology  that  both  indicators 
and  means  of  verification  be  developed  as  part  of  any  evaluation. 
(Schmidt  et  al. ,  1977) 

The  implication  of  these  statements  is  that  performance 
measurement  can  and  should  oe  done  using  management  criteria  as  a 
basis  for  evaluation.  This  type  of  approach  to  program  evaluation  is 
known  as  "goal  achievement"  (Franklin  and  Thrasher,  1976).  The 
advantages  to  the  goal  achievement  approach  is  thar  it  allows  the 
implementor  to  be  the  control  for  the  study  and  it  insures  that  the 
evaluation  is  measurable  and  relevant  to  the  implementor 's  original 
problems  (Franklin  and  Thrasher,  1976).  This  approach  is  particularly 
useful  in  the  evaluation  of  innovative  programs  (Rossi  and  Freeman, 
1982).  Office  automation  systems  in  Naval  Laboratories,  specifically 
the  ATIPS  system,  are  innovative.  Therefore,  this  thesis  uses  the  goal 
achievement  approach  in  its  measurement  of  the  performance  of  the 
ATIPS  program  at  NWC  China  Lake. 

Rossi  and  Freeman  (1982)  write  that  the  evaluation  of  an 
innovative  program  should  include: 

1.  Identifying  and  describing  the  problem  of  concern, 

2.  Identifying  the  program  and  delineating  the  objectives  developed 
to  solve  the  problem, 

3 .  Designing  an  impact  model , 

4.  Designing  an  evidence  gathering  technique. 
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This  chapter  follows  this  basic  outline.  The  outline  contains  terms 
and  concepts  that  may  not  be  intuitively  obvious;  these  terms  and  the 
concepts  in  each  step  are  explained  as  they  are  addressed. 

A.  PROBLEM  IDENTIFICATION  AND  DESCRIPTION 

The  following  paragraph  is  taken  from  the  ATIPS  Concept 

Development  Paper  (NWC,  March  1982): 

The  Naval  Weapons  Center  (NWC),  China  Lake,  faces  heavily  increasing 
administrative  workloads  along  with  reductions  in  tne  manpower 
currently  handling  those  workloads.  NWC  personnel  work  in  an  almost 
exclusively  manual  administrative  environment.  What  is  needed  is  a 
more  efficient  methodology  for  handling  the  necessary  administrative 
tasks  of  Center  personnel,  (p.  1) 

An  Inspector  General's  inspection  of  NWC  in  March  1980  reported 
that  manpower  ceilings  had  resulted  in  an  increase  in  personnel 
workload  and  a  decrease  in  the  quality  of  work  (NWC,  July  1980). 
Through  the  1970s  the  workforce  at  NWC  had  been  reduced  twenty  percent 
due  to  billet  restrictions  while  the  total  budget  and  workload  had 
increased  twenty  percent  (NWC,  July  1980). 

The  problem  that  faced  management  at  NWC  China  Lake  was  how  to  do 
more  work  with  less  people  without  negatively  effecting  the  quality  of 
the  end  product. 

B.  PROGRAM  AND  OBJECTIVES 

In  response  to  the  problems  listed  above,  four  alternative  courses 
of  action  were  investigated;  the  alternative  that  premised  the  highest 
return  on  investment  in  the  shortest  period  of  time  was  the 
coordinated  development  of  an  Automated  Technical  Information 
Processing  System  (ATIPS)  (NWC,  August  1981).  As  discussed  previously, 
ATIPS  is  an  office  automation  system  that  uses  a  central  computer  and 
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remote  access  terminals  to  provide  a  variety  of  functions  to  the 

users.  Among  these  functions  are  (NALTOACS,  1983): 

word  processing       electronic  mail 
document  processing    electronic  calendar 
business  graphics      form  processing 

The  objectives  of  ATIPS  were  delineated  in  the  Concept  Development 

Paper  (NWC,  August  1981).  These  objectives  are  listed  below: 

1.  Reduce  administrative  workload  for  technical  and  administrative 
personnel . 

2.  Reduce  the  amount  of  space  required  to  store  information. 

3.  Reduce  the  costs  for  typing,  copying,  transmitting,  filing,  and 
retrieving  information. 

4.  Reduce  the  number  and  size  of  meetings. 

5.  Reduce  the  labor  cost  for  each  productive  unit  of  work. 

6.  Make  NWC  a  more  cost-effective  Research,  Development,  Test,  and 
Evaluation  Center. 

7.  Provide  access  to  other  NWC  computers  through  a  single  terminal. 

8.  Provide  a  direct  communication  route  among  all  personnel  who 
require  it. 

9.  Provide  ready  access  to  large  quantities  of  textual  information. 
Another  objective  of  the  system  (Objective  10)  that  was  not 
communicated  in  the  Concept  Development  Paper  but  was  communicated  to 
the  author  by  the  functional  manager  involved  the  quality  of  the 
deliverables  that  the  Center  produced;  ATTPS  was  to  increase  the 
quality  of  the  reports,  manuals,  briefings,  and  documents  that  the 
Center  produced. 

Greenwood's  (1984)  contention  that  productivity  is  a  measure  of 
efficiency  and  effectiveness  can  be  applied  to  the  stated  ATIPS 
objectives.  Efficiency  is  the  comparison  of  inputs  to  outputs;  the 
fewer  inputs  per  unit  of  output  (or  the  greater  output  per  unit  of 
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input)  identifies  an  increase  in  efficiency.  The  first  seven 
objectives  listed  for  ATIPS  are  questions  of  efficiency;  each  of  these 
objectives  seeks  to  maximize  outputs  while  minimizing  inputs.  The  last 
three  objectives  are  not  concerned  directly  with  efficiency  (although 
that  may  be  a  side  effect),  rather  these  objectives  are  questions  of 
effectiveness,  of  how  well  each  worker  and  the  Center  can  achieve  its 
overall  goal. 

The  basic  objective  of  ATIPS  can  be  reduced  to:  increasing 
personal  and  Center  productivity  througn  increasing  both  efficiency 
and  effectiveness. 

C.   IMPACT  MODEL 

An  impact  model,  which  is  similar  to  a  logical  model  (Wholey, 
1979),  is  an  attempt  to  translate  notions  regarding  the 
modifications  and  controls  of  a  condition  into  a  hypotheses  on 
which  action  can  be  based  (Rossi  and  Freeman,  1982).  Tnere  are  two 
hypotheses  that  should  be  addressed: 

1.  the  causal  hypothesis, 

2.  the  intervention  hypothesis  (Rossi  and  Freeman,  1982). 

The  causal  hypothesis  is  a  hypothesis  about  the  influence  of  one  or 
more  processes  on  the  condition  that  the  program  seeks  to  modify.  It 
is  the  relationship  between  the  program  or  system  inputs,  activities, 
and  outcomes  (Wholey,  1979).  For  NWC  China  Lake  the  causal  hypothesis 
is:  The  availability  of  only  manual  administrative  tools  coupled  with 
an  increasing  workload  and  a  decreasing  workforce  is  responsible  for 
lower  productivity  at  NWC  China  Lake. 

Using  this  causal  hypothesis  as  a  base,  the  remainder  of  the 
impact  model  can  be  developed  through  the  intervention  hypothesis 
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(Rossi  and  Freeman,  1982).  The  intervention  hypothesis  is  a  statement 

that  specifies  the  relationship  between  a  program,  what  is  going  to  be 
done,  and  the  causal  hypothesis.  In  the  case  of  NWC  China  Lake  there 

are  three  variables  (manual  administrative  tools,  increasing  workload, 

and  decreasing  work  force)  in  the  causal  hypothesis  that  influence  the 

outcome  (productivity).  The  only  means  by  which  to  change  the  outcome 

is  by  changing  one  of  the  three  variables.  However,  it  was  not 

possible  for  the  Center  to  hire  more  workers  or  to  refuse  projects  and 

lighten  its  workload.  Therefore,  the  first  two  variables  are 

constants.  The  only  variable  in  which  change  was  feasible  was  the  use 

of  manual  administrative  tools.  ATIPS  was  purchased  to  replace  manual 

administrative  tools  with  automated  tools.  The  goal  was  to  increase 

worker  productivity.  Therefore,  the  intervention  hypothesis  is:  Office 

automation  technology  increases  overall  productivity  at  NWC  China 

Lake. 

The  impact  or  logical  model  provides  a  clear  view  of  the 

relationships  between  the  initial  conditions  and  problems,  the 

intervening  program,  and  the  outcomes. 

D.   EVIDENCE  GATHERING/MEASUREMENT  MODEL 

The  evidence  gathering  technique  that  is  used  in  an  evaluation  is 
influenced  by  the  basic  evaluation  methodology  and  the  evaluative 
approach.  The  basic  evaluation  methodology  that  this  proposed  OA 
cost/benefits  analysis  method  uses  is  the  quantitative  method 
described  in  chapter  two.  The  evaluative  approach  that  this  proposed 
OA  cost/benefits  analysis  method  uses  is  the  goal  achievement  approach 
described  earlier  in  this  chapter.  The  implication  of  using  the 
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quantitative  methodology  and  the  goal  achievement  approach  is  that  the 
evidence  gathering  technique  should  yield  ostensive  data  for 
comparative  analysis  by  utilizing  the  program's  objectives  and  the 
program  management's  projected  performance  measurement  criteria. 
Therefore,  the  project  ATIPS  performance  criteria  and  the  program's 
objectives  are  examined  prior  to  the  selection  of  an  evidence 
gathering  technique.  Once  the  evidence  gathering  technique  is 
determined  the  measures  of  program  performance  are  refined  from  the 
projected  program  performance  criteria  and  the  program's  objectives. 
Following  that  refinement  evidence  gathering  technique  can  be 
implemented . 

1.  Projected  ATIPS  Performance 

NWC  contracted  The  Mattox  Group,  a  consulting  firm  in 
Pasadena,  California,  to  study  a  portion  of  the  workers  at  the  Center 
for  the  purpose  of  identifying  potential  opportunities  to  increase  the 
efficiency  and  effectiveness  of  the  Center 's  managerial ,  technical , 
and  support  personnel  (Mattox,  February  1981).  The  study  consisted  of 
three  parts.  First,  a  "shadowing"  was  conducted  of  each  member  of  a 
pilot  group  for  8  to  16  hours.  Second,  250  representatives  from  all 
departments  were  interviewed.  Third,  interview  data  was  correlated 
with  the  "shadow"  data  (see  Appendix  S  for  Mattox  Group  evaluation 
data).  The  Mattox  Group  study  concluded  that  an  office  automation 
system  at  the  Center  could  have  a  tremendous  impact  and  potentially 
save  managers,  technical  and  support  staff  20%  of  rheir  time. 

The  ATIPS  project  development  group  used  this  20%  figure  as  a 
basis  to  project  the  potential  personnel  salary /wage  cost  avoidance 
savings  that  could  be  expected  throughout  the  life  of  ATIPS  (see 
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Appendix  B  for  the  figures  and  calculations).  The  analysis  indicated 
that  ATIPS  would  result  in  a  $5.7  M  personnel  cost  avoidance  in  its 
first  year  of  operation.  This  figure  would  increase  as  ATIPS  grew  to 
its  full  size  and  the  life  cycle  cost  avoidance  for  personnel  could 
reach  $151  M  (NWC,  August  1981). 

The  second  area  of  quantitative  measurement  criteria  involves 
travel.  The  Mattox  Group  estimated  that  a  15%  reduction  in  travel 
requirements  would  be  possible  through  the  use  of  ATIPS  communication 
capabilities  (Mattox,  February  1981).  NWC  personnel  were  averaging 
7000  off  Center  trips  per  year  at  an  average  cost  of  $1,200  per  trip 
(NWC,  August  1981).  This  meant  that  ATIPS  could  save  $420  K  in  its 
first  year  of  operation  and  $8.82  M  over  its  life-cycle  (see  Appendix 
B) .  In  addition  to  the  reduction  in  the  cost  of  travel  the  personnel 
cost  avoidance  that  resulted  from  the  decreased  travel  time  was 
calculated.  This  would  result  in  a  $70  K  cost  avoidance  in  the  first 
year  of  ATIPS  operation  and  a  $1.4  M  cost  avoidance  over  the  life- 
cycle  (NWC,  August  1981)  (see  Appendix  B). 

The  project  group  also  estimated  that  50%  of  the  Center 's 
costs  for  maintenance  of  office  equipment  would  be  avoided  due  to  less 
wear  and  decreased  usage  of  copiers,  typewriters,  and  other  office 
equipment.  This  would  result  in  a  $200  K  savings  the  first  year  of 
ATIPS  operation  and  a  $4.2  M  savings  over  the  life-cycle  of  the 
system  (NWC,  August  1981)  (see  Appendix  B) . 
2.  ATIPS  Program  Objectives 

The  goal  achievement  method  of  performance  measurement 
requires  that  tne  evidence  gathering  technique  for  a  program  be  based 
upon  the  management 's  measurement  criteria  and  the  program 's 
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objectives  and  goals  (Franklin  and  Thrasher,  1976).  As  presented 
previously,  the  ATIPS  program  objectives  are: 

1.  Reduce  administrative  workload  for  technical  and  administrative 
personnel . 

2.  Reduce  the  costs  for  typing,  copying,  transmitting,  filing,  and 
retrieving  information. 

3.  Reduce  the  labor  cost  for  each  productive  unit  of  work. 

4.  Reduce  the  number  and  size  of  meetings. 

5.  Reduce  the  amount  of  space  required  to  store  information. 

6.  Provide  access  to  other  NWC  computers  through  a  single  terminal. 

7.  Provide  a  direct  communication  route  among  all  personnel  who 

require  it. 

8.  Provide  ready  access  to  large  quantities  of  textual  information. 

9.  Increase  the  quality  of  the  deliverables  produced  by  the  Center. 

10.  Make  NWC  a  more  cost-effective  Research,  Development,  Test,  and 
Evaluation  Center. 

3.  Evaluation  Technique  Selection 

The  projected  ATIPS  performance  criteria  and  the  ATIPS  program 

objectives  are  similar  in  that  they  both  suggest  that  the  introduction 

of  ATIPS  will  result  in  either  reductions  or  increases  of  costs, 

workloads,  cost-effectiveness,  and  quantities  of  textual  information, 

for  example.  This  suggests  that  measurements  made  before  the 

introduction  of  ATIPS  could  be  compared  to  measurements  made  after  the 

introduction  of  ATIPS;  this  comparison  would  result  in  indication  of 

benefits  realized/not  realized  due  to  the  introduction  of  ATIPS.  This 

type  of  evidence  gathering  technique  is  known  as  a  quasi-experimental 

time  series  design  (Franklin  and  Thrasher,  1976).  Wholey  (1979) 

describes  the  quasi -experimental  time  series  design  of  evidence 

gatnering. 
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.  .  .  the  evaluator  measures  outcome  variaoles  several  times  before, 
and  several  times  after,  participants  are  exposed  to  the  program. 
Program  effectiveness  is  estimated  by  comparing  the  post-program 
measurements  with  the  pre-prcgram  measurements.  The  evaluator  plots 
the  time  series  data;  tnen  the  evaluator  and  other  knowledgeable 
people  examine  the  data,  consider  possible  alternative  explanations 
for  the  observed  results,  and  attempt  to  draw  conclusions  about  the 
effectiveness  of  the  program,  (p.  160) 

This  evidence  gathering  technique  fits  the  with  the  projected  ATIPS 

performance  criteria  and  the  objectives  listed  for  the  ATIPS  program. 

The  qua si -experimental  time  series  design  is  the  evidence  gathering 

technique  that  this  proposed  evaluation  method  uses. 

4.  Refinement  of  Program  Performance  Measures 

Goal  achievement  is  the  evaluative  approach  used  for  the  OA 

benefits  evaluation  method  proposed  by  this  thesis.  As  stated  earlier, 

the  goal  achievement  approach  requires  that  the  program  objectives  and 

the  projected  program  performance  measures  be  used  as  the  basis  for 

the  evaluation.  This  means  that  the  selection  of  specific  performance 

measures  should  involve  the  program's  objectives  and  projected 

performance  measures. 

The  ATIPS  program  objectives  are  listed  on  page  9  of  this  chapter. 

These  10  objectives  can  be  combined  into  five  groups  of  objectives: 

1 .  Reduce  the  cost  of  administration  by  reducing  the  workload  and 
reducing  the  cost  per  unit  of  output. 

2.  Allow  access  through  computer  terminals  to  multiple  computers, 
large  text  libraries,  and  otner  personnel. 

3.  Reduce  the  number  of  meetings. 

4.  Reduce  storage  space  needed  for  information. 

5.  Increase  the  quality  of  the  work  produced. 

The  ATIPS  program's  projected  performance  criteria  are 
explained  on  pages  7  and  8  of  this  chapter.  These  performance  criteria 
list  three  areas  of  performance  measurement: 
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1.  personnel  cost  avoidance, 

2.  reduction  of  meetings  requiring  travel  away  from  NWC  China  Lake, 

3.  reduction  in  the  cost  of  maintenance  on  office  equipment. 

How  do  the  objectives  relate  to  the  performance  measurement  criteria 
established  by  management  prior  to  the  introduction  of  ATIPS?  There 
appears  to  be  only  one  direct  relationship,  that  is  between  the 
objective  to  reduce  meetings  and  the  projected  performance  criteria  to 
reduce  travel  due  to  meetings.  However,  this  projected  performance 
measurement  does  not  include  meetings  that  are  held  at  NWC  China  Lake. 

The  cost  avoidance  performance  measurement  is  only  loosely 
related  to  the  objectives  of  cost  reduction  of  administration  and 
reduction  of  the  number  of  meetings.  Cost  avoidance  as  a  performance 
measure  does  not  lend  itself  to  true  measurement  after  the  ATIPS 
program  is  in  place.  This  is  in  contrast  to  the  other  two  projected 
performance  criteria  which  do  lend  themselves  to  measurement. 
Therefore,  the  author  views  cost  avoidance  as  more  of  a  goal  of  the 
ATIPS  system  rather  than  a  performance  measure  for  the  program's 
stated  goals. 

The  question  remaining  is  wnat  types  of  measures  of 

performance  should  be  used  to  provide  evidence  about  the  benefits 

obtained  from  the  incorporation  of  the  ATIPS  system?  This  thesis  can 

only  offer  suggestions  as  to  the  appropriate  measures  of  performance 

because  to  select  these  measures  requires  the  involvement  of  the 

management  (Rossi  and  Freeman,  1982).  Wholey  (1979)  wrote: 

The  key  to  useful  performance  monitoring  is  agreement  between  the 
evaluator  and  the  intended  user  on  the  events  to  be  monitored,  tne 
measures  to  be  used  in  the  monitoring,  and  the  intended  uses  of  the 
resulting  information,  (p.  137) 
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This  information  leads  the  author  of  this  thesis  to  suggest 
appropriate  performance  measures  for  each  of  the  oojectives  of  the 
ATIPS  program.  Each  of  the  objective  groupings  on  page  9  are  addressed 
in  order. 

To  measure  the  effect  of  ATIPS  on  the  volume  of  paperwork  at 
NSvC  (Objective  1)  paper  itself  should  be  measured.  A  comparison  of 
paper  usage  should  yield  a  reduction  in  usage  after  ATIPS  was 
introduced.  This  measure  should  be  weighted  to  account  for  the 
differences  in  project  load  that  SJWC  experiences  during  tne 
measurement  periods. 

To  measure  the  the  effect  of  ATIPS  on  the  cost  of 
administrative  work  (Objectives  2  and  3)  a  comparison  of 
administrative  output  and  cost  should  oe  made.  Administrative  output 
is  not  synonymous  with  total  paper  usage.  Output  is  the  sum  of  all  tne 
manuals,  reports,  briefings,  presentations,  letters,  and  the  like  that 
;NKC  China  Lake  produces  during  a  period  of  time.  Cost  is  the  total 
equipment  maintenance  and  labor  costs  during  the  same  period  of  time. 
Labor  costs  are  particularly  difficult  to  measure  because  of  the 
concept  of  cost  avoidance.  It  is  unlikely  that  there  will  be  any 
savings  in  personnel  labor  costs  per  se.  However,  the  concept  of  cost 
avoidance  implies  that  personnel  will  complete  tasks  in  a  more  timely 
fashion.  The  implication  of  this  statement  is  that  personnel  should  be 
able  to  perform  more  tasks  in  the  same  unit  of  time.  Therefore,  since 
M\7C  is  not  expected  to  reduce  its  worK  force,  to  detect  a  change  in 
labor  cost  per  unit  of  work  will  require  measuring  workload  and  staff 
size:  the  output  should  go  up  while  tne  labor  force  remains  constant. 
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To  measure  the  effect  that  ATIPS  has  on  the  number  and  the 
size  of  meetings  (Objective  4)  the  number  of  off  station  trips  made 
due  to  meetings  can  be  measured  and  compared  (as  suggested  by  the 
projected  performance  measurement  criteria).  Also,  historical  records 
could  be  used  to  determine  the  number  of  on  station  meetings  required 
before  ATIPS;  the  number  of  meeting  required  since  the  installation  of 
ATIPS  could  be  measured  and  a  comparison  made. 

A  measure  of  the  amount  of  space  required  to  store  information 
(Objective  5)  could  involve  researching  supply  records  to  obtain 
information  concerning  the  purchase  of  storage  units  before  and  after 
the  incorporation  of  ATIPS. 

Measurement  of  the  access  to  other  computers  tnrough  ATIPS 
terminals,  measurement  of  communications  capabilities,  and  the  amount 
of  textual  information  available  through  an  ATIPS  terminal  (Objectives 
6-8)  can  all  be  measured  directly  without  research. 

Measurement  of  the  quality  of  the  deliverables  produced  by  the 
center  (Objective  9)  before  and  after  ATIPS  could  be  done  by  direct 
comparison.  Copies  of  deliverables  produced  before  ATIPS  incorporation 
could  be  compared  with  deliverables  that  are  similar  in  nature  but 
produced  after  ATIPS  incorporation.  The  judges  of  the  quality  of  the 
deliverables  could  be  a  composite  team  of  professionals  from  both 
outside  and  within  NWC  China  Lake.  Stuff lebeam  et  al.  (1974)  wrote 
that  professional  judgement  is  an  effective  evaluation  tool  when 
measurement  criteria  are  difficult  to  define  and  quantify. 
Professional  judgement  is  based  on  the  evaluators  experience  and 
expertise.  The  criteria  of  the  quality  measurement  should  be  agreed 
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upon  by  the  professionals  prior  to  the  actual  performance  of  the 
evaluation. 

The  final  objective  is  an  objective  that  encompasses  all  of 
the  others  and  includes  the  purpose  of  this  proposed  evaluation 
method:  making  NWC  more  cost  effective.  Measurement  of  the  other 
objectives  will  provide  evidence  for  this  objective.  This  is  where  the 
issue  of  cost/benefit  is  resolved.  The  measures  described  above 
produce  figures  of  relative  savings.  These  relative  savings  should  be 
totaled  and  compared  to  the  cost  of  the  system.  If  the  savings  total 
is  greater  than  the  cost  of  the  system  then  the  system  is  worth 
purchasing;  it  is  worth  purchasing  for  all  the  Naval  Laboratories. 

E.   SUMMARY 

This  chapter  describes  an  method  of  evaluating  the  impact  of 
office  automation  technology  in  Naval  Laboratories.  The  Naval  Weapons 
Center  China  Lake,  California  is  the  subject  for  a  case  study;  by 
utilizing  information  from  NWC  an  evaluation  method  is  presented  using 
the  qualitative  methodology  and  the  goal  achievement  evaluative 
approach.  The  goal  of  the  evaluation  method  is  to  provide  ostensive 
data  for  the  performance  of  a  cost/benefit  analysis. 
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IV.   SUMMARY  AND  RECOMMENDATIONS 

This  thesis  presents  a  method  for  determining  the  productivity 
effects  and  the  cost/benefit  of  office  automation  in  Naval 
Laboratories.  Formulation  of  this  method  included  analysis  of  the 
NALTOACS  Pilot  Assessment  and  an  analysis  of  the  pilot  office 
automation  system  (ATIPS)  at  NWC  China  Lake,  California. 

Chapter  I  presents  a  definition  of  office  automation  and  provides 
evidence  that  organizations  can  realize  both  tangible  and  intangible 
benefits  through  the  incorporation  of  OA  technology.  The  primary 
benefit  claimed  for  OA  is  increased  productivity.  However,  a  survey  of 
current  literature  indicated  skepticism  over  the  realization  of 
benefits  due  to  OA.  The  chapter  concludes  with  an  overview  of  the 
NALTOACS  Pilot  Assessment  and  the  ATIPS  system  at  NWC  China  Lake. 

Chapter  II  analyzes  the  NALTOACS  Pilot  Assessment  method  in 
detail.  A  qualitative  method  was  used  to  perform  the  NALTOACS  Pilot 
Assessment.  The  chapter  provides  evidence  showing  that  this 
methodology  was  inappropriate  for  two  reasons.  First,  the  results  of 
the  qualitative  evaluation  are  based  on  opinions  which  are  not 
ostensive  and  confirmable  in  nature.  Second,  the  qualitative 
methodology  can  produce  erroneous  results  due  to  the  biases  inherent 
in  this  evaluative  paradigm. 

Chapter  III  presents  a  method  of  evaluating  the  impact  of  office 
automation  in  Naval  Laboratories,  a  method  based  on  a  quantitative 
method.  This  evaluation  method  incorporates  the  goal-achievement- 
evaluative  approach  and  uses  the  ATIPS  system  at  NWC  China  Lake  as  a 
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subject  of  a  case  study.  The  goal  of  the  evaluation  method  is  to 
provide  ostensive  data  for  the  performance  of  a  cost/benefit  analysis. 
The  purpose  of  using  the  goal -achievement-evaluative  approach  is  to 
insure  that  the  goals,  objectives,  and  performance  criteria  developed 
during  project  initiation  are  used  in  the  project  evaluation. 

The  ATIPS  office  automation  program  was  estimated  to  cost  nearly 
$12  million.  Incorporation  of  OA  systems  at  all  eight  Naval 
laboratories  cculd  cost  more  than  $100  million.  One  performance 
criterion  for  the  investment  of  public  funds  is  that  the  trustees  of 
those  funds  maximize  the  return  on  the  investment;  this  is 
responsibility  of  the  position  and  demanded  by  the  public  (Franklin 
and  Thrasher,  1976).  To  ensure  that  goal  of  maximizing  return  on 
investment  is  achieved,  the  implementation  of  evaluation  techniques  is 
required.  These  evaluation  techniques  should  produce  measurable  and 
confirmable  results. 

The  NALTOACS  Pilot  Assessment  was  a  qualitative  evaluation  of 
office  automation  in  Naval  Laboratories.  The  evaluation  results  were  a 
compilation  of  the  opinions  of  the  surveyed  respondents;  inferences 
about  benefits  and  productivity  gains  due  to  OA  were  made  based  on 
those  opinions.  The  actual  productivity  results  were  not  measured  nor 
confirmed  nor  did  they  appear  to  be  measurable  or  confirmable.  Hence, 
the  NALTOACS  Pilot  Assessment  does  not  provide  information  which  is 
useful  in  determining  the  program's  return  on  investment. 

The  evaluation  method  proposed  by  this  thesis  produces  results 
that  are  both  measurable  and  confirmable.  Further,  the  data  produced 
by  the  evaluation  method  insures  that  the  objectives  of  the  system  are 
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considered  in  its  evaluation  and  that  the  program's  cost  effectiveness 
is  determined. 

NALTOACS  was  established  to  coordinate  the  introduction  of  OA 
systems  as  means  to  enhance  productivity  in  Naval  Laboratories 
(NALTOACS,  1983).  The  recommendation  of  the  author  is  to  perform  an 
evaluation  of  an  operating  Naval  Laboratory  OA  system  using  the  method 
described  in  this  thesis.  This  will  provide  ostensive  data  which  will 
indicate  whether  office  automation  does  indeed  enhance  productivity 
and  whether  that  enhancement  is  cost  effective. 
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APPENDIX  A 
NALTOACS  PILOT  ASSESSMENT  USER  INTERVIEW  CHECKLIST 
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NALTOACS  PTT.OT  ASSESSMENT 
USER  INTERVIEW  CHECKLIST  PACKAGE 

The  data  being  collected  by  this  survey  is  required  to  conduct 
the  "Pilot  Assessment"  directed  by  the  Deputy  Under  Secretary  of 
the  Navy  (Financial  Management)  and  the  Commander,  Naval  Data 
Automation  Command. 

Please  take  this  opportunity  to  candidly  express  your 
opinions  and  needs  so  that  the  overall  planning  for  office 
automation  (OA)  in  the  Naval  laboratories  can  directly  benefit 
from  them.  Also,  please  feel  free  to  add  any  inputs  which  you 
may  have  and  which  are  not  covered  by  the  checklist. 

Thank  you  for  your  cooperation  and  assistance. 


Project  Manager 


USER   PROFILE 
Name: 


Code:. 


Telephone: 


Bldg:. 


Date: 


How  would  you  classify  your  job  responsibilities: 
Command  and  Support  Technical 


Executive 

Manager 

Supervisor 

Professional 

Clerical 

Other  


Manager 
Scientist 
Systems  Engineer 
Prog rammer/ Analyst 
Other  


SURVEYOR 


TELE  # 


NWC 


1.  Did  you  use  other  keyboard  devices  in  your  office  before  the 
pilot  OA  was  implemented? 

No —  Yes__                Still  use_ 

Typewriter        Typewriter        

Computer  Terminal  Computer  Terminal  

WP  Workstation    WP  Workstation 

Personal  Computer  Personal  Computer  

2.  Do  you  have  a  personal  computer  at  home? 
No Yes 

3.  Do  you  have  your  own  pilot  OA  workstation? 

No — ,  how  far  away  is  the  workstation  you  share?   ft. 

Yes ,  with 

Dedicated  Letter  Quality  Printer 

Shared  Letter  Quality  Printer    How  far  away? ft. 

Dedicated  Dot  Matrix  Printer 

Shared  Dot  Matrix  Printer        How  far  away? ft. 

Shared  Plotter  How  far  away? ft. 

Local  Storage 

4.  Do  you  need  your  own  dedicated  OA  workstation? 
No Yes 

5.  Do  you  or  would  you  use  it  as  a  terminal  for  programming? 
No Yes 

6.  How  long  have  you  been  an  active  user  of  the  pilot  OA  system? 

Less  than  1  month    Over  6  months 

Less  than  3  months   Over  12  months 

Less  than  6  months   Over  18  months 

7.  Do  you  feel  that  you  were  adequately  prepared/trained  to  use 
the  OA  system? 

No Yes Did  not  attend  training  sessions 

8.  Do  you  feel  the  system  provides  enough  OA  self-help  features 
to  enable  you  to  use  it  with  ease? 

No Yes  ,  for  frequently  used  capabilities 

No Yes  ,  for  infrequently  used  capabilities 


i   o 


).   Do  you  feel 
assistance? 


the  pilot  project  staff  provides  adequate  user 


No Y*s 


10.  The  following  are  general  descriptions  of  the  basic  OA 
functional  capabilities  included  in  some  pilot  OA  systems. 
After  each  is  a  series  of  questions  pertaining  to  your 
knowledge,  use  and  assessment  of  them.  (Please  check  the 
appropriate  response/responses.) 

CALENDAR  -  Allows  a  user  to  maintain  an  appointment  calendar, 
both  public  and  private.  It  provides  the  functions  for 
adding,  changing,  deleting,  searching  and  viewing  the 
entries. 


.Essential 
.Nice  to  have 
Do  not  need 


.Available 
.Not  available 
.Do  not  know  if 
available 


_Ose  frequently 
.Use  occasionally 
.Do  not  use 


I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 


.Adequate  to  my  needs 
.Must  be  improved, 
how  so  


.Easy  to  use 
.Difficult  to  use, 
how  so  


TICKLER  Notifies  the  user  on  a  given  day  and  time  that  a 
certain  action  or  task,  previously  entered  by  the  user,  needs 
to  be  performed  that  day. 


.Essential 
.Nice  to  have 
Do  not  need 


.Available 
.Not  available 
.Do  not  know  if 
available 


.Use  frequently 
.Use  occasionally 
Do  not  use 


I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 


.Adequate  to  my  needs 
.Must  be  improved, 
how  so  


.Easy  to  use 
.Difficult  to  use, 
how  so  


-  Provides  the  user  with  a  standard  telephone 
message  pad  for  taking  telephone  messages  for  other  users  on 
the  system.  The  message  will  be  sent  to  the  recipient 
electronically. 


.Essential 
.Nice  to  have 
Do  not  need 


.Available 
.Not  available 
.Do  not  know  if 
available 


.Use  frequently 
.Use  occasionally 
Do  not  use 
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I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs         Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so  how  so  


DIRECTORY  -  Provides  a  convenient  way  for  looking  up  the 
telephone  numbers  of  associates  that  are  frequently  called. 

Essential      Available  Use  frequently 

Nice  to  have   Not  available  Use  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 

I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs  Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so how  so  


BULLETIN  BOARD  -  Announces  important  events  or  information  of 
general  interest  to  all  or  a  specified  group  of  users. 
Announcements  may  refer  to  a  longer  explanatory  record  that 
can  be  retrieved  by  interested  parties. 

Essential      Available  Use  frequently 

Nice  to  have   Not  available  Use  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 

I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs         Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so  how  so  


ELECTRONIC  mail  -  Replaces  the  inter-office  memoranda  and 
other  informal  correspondence.  It  allows  the  user  to  scan 
his  mail  box,  to  read  mail,  to  reply  to  the  mail,  and  to  file 
and  print  mail. 

Essential      Available  Use  frequently 

_Nice  to  have   Not  available  Use  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 
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I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs  Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so  how  so  


WORD  PROCESSING  -  Provides  a  comprehensive  input  and  editing 
capability  that  allows  users  to  create,  edit  and  manipulate 
text  quickly  and  easily. 

Essential      Available  Use  frequently 

Nice  to  have   Not  available  Dse  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 

I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs  Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so  how  so  


FORM  processing  -  Allows  the  user  to  retrieve  a  standard 
form,  insert  the  information  required  and  then  store,  print 
or  forward  the  document.  The  user  may  also  create  special 
forms  for  one-time  or  repetitive  use. 

Essential      Available  Use  frequently 

Nice  to  have   Not  available  Use  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 

I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs         Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so  how  so  


DOCUMENT  PROCESSING  -  Extension  of  the  basic  word  processing 
system  to  permit  the  pulling  together  of  text  from  various 
sources  to  create  a  large  document  in  a  consistent  format. 

Essential      Available  Use  frequently 

Nice  to  have   Not  available  Use  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 


SCHEDULER  -  Determines  the  earliest  time  that  all  specified 
attendees  are  available  by  checking  their  individual 
calendar  records.  Once  all  attendees  have  confirmed  the  date 
and  time,  the  meeting  will  be  scheduled  and  an  entry  will 
appear  in  each  user's  calendar. 

Essential      Available  Use  frequently 

Nice  to  have   Not  available  Use  occasionally 

Do  not  need    Do  not  know  if  Do  not  use 

available 

I  have /have  not been  trained  on  this  capability  and 

I  would  classify  the  capability  as: 

Adequate  to  my  needs  Easy  to  use 

Must  be  improved,  Difficult  to  use, 

how  so  how  so  


11.  How  do  you  rate  the  overall  responsiveness  of  the  system  in 
operation? 

Fast  (under  2  sec)  Slow  (5-10  sec) 

Adequate  (2-4  sec)  Too  Slow  (10  sec  or  above) 

12.  In  what  OA  capabilities  has  response  time  been  the  slowest 
(list  slowest  first)? 

1. 3. 

2. 4. 


13.  On  the  average,  what  percent  of  your  day  is  spent  using  your 
OA  workstation? 

0-5%    (1/2  hour) 

6-12%   (1  hour) 

13-25%  (2  hours) 

26-37%  (3  hours) 

38-50%  (4  hours) 

Over  50% 


14.  Which  capabilities  do  you  use  the  most? 

1 

2._ 

3.__ 

4. . 
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15.  Which  capabilities  would  you  use  more  extensively  if  there 
were  more  users  on  the  system? 

1«— EXAMPLES:   Electronic  Mail, 

Scheduler,  Bulletin  Board, 
2. etc. 

3. 

4._ 


16.  What  specific  functions  or  products  do  you  use  the  system  to 
complete? 

1. EXAMPLES:   Correspondence, 

Technical  Reports,  Speci- 

2. fications,  Memoranda,  Admin- 
istrative Reports  (by  name) , 

3. Statement  of  Work,  Procure- 
ment Requests,  etc. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 


17.  In  what  way  does  the  system  assist  you  in  the  preparation  of 
these  products? 

Get  the  job  done  with  less  effort,  how  much  so? % 

Better  quality  of  end  product,  how  so? 

Fewer  typos 

Better  appearance 

More  timely 

Other  (Specify) 
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18.    The    following   are   general    descriptions    of    additional   OA 
functional    capabilities    that  may  be   installed   in  the   future 
on    your     OA    system.       Please    check    the    appropriate 
response/ responses. 

CORRESPONDENCE  CONTROL  -  Enables  the  tracking  of  formal 
correspondence  as  it  moves  from  one  addressee  (location)  to 
the  next.  It  requires  the  posting  of  receipt  and  forwarded 
entries  as   it  moves   from  one  point  to  another. 

Essential;   would  use   frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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phonebook  -  Provides  a  central  telephone  directory-like 
listing  available  to  all  users  which  includes  the  name, 
telephone  number,  organizational  code,  and  office  location  of 
all  on-base  personnel. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


PERT  -  Allows  a  user  to  construct  and  maintain  PERT  networks, 
and  do  CPM  analyses. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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INTERACT  -  Allows  users  to  send  and  receive  messages  to  each 
other  in  a  dialogue  fashion  through  their  workstations. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If    this    capability    is    made   available,    specify  which  office 
products    and    their    associated    tasks   would   be    accomplished 
more   efficiently. 

Products  Tasks 


ROUTING    -    Allows    an    electronic    document    to    be    routed 
sequentially    to    other    users  according   to  an   input   specified 
or   pre-established  distribution  list. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


TNTERCQNNECT  -  Allows  the  workstation  to  be  linked  to  a 
computer  system  such  that  the  workstation  becomes  a  terminal 
on  that  system  for  programming,  data  retrieval,  etc. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If   this   capability   is   made  available,    specify  which  office 
products    and   their    associated   tasks   would   be   accomplished 
more   efficiently. 

Products  Tasks 


SIGNATURE  -  Allows  a  user  to  "sign"  a  document  electronically 
by  typing  in  an  alphanumeric  string  known  only  to 
himself/herself   and  the   system. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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audible  matt.  -  Allows  the  user  to  listen  to  a  computer 
readout  of  his  electronic  mail,  when  the  user  is  away  from 
his/her  workstation  but  has  access  to  a  touchtone  telephone. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


VOICE  MAIL  -  An  audio  version  of  electronic  mail.   Messages 
are  input  to  the  system  by  voice  and  recordings  are  retrieved 
by  the  addressees. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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TELECONFERENCING  -  A  form  of  computer  session  where  all 
attendees  are  simultaneously  connected  and  communicate  with 
each  other  in  an  interactive  mode. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If    this   capability   is   made   available,    specify  which  office 
products   and   their   associated   tasks  would  be   accomplished 
more  efficiently. 

Products  Tasks 


STOPLIST  -   Points   out   all   occurrences  of  words   in  the  text 
which  have  been  prohibited. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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ARCHIVE   -   Allows   documents   to  be   filed  with  user-specified 
filing    codes,    keywords,    and    indexes,    such    as    subject, 
originator,    file    number,    serial   number   and  date.      Users   can 
retrieve    a    copy    or    restore    the    documents    to   active    (and 
modifiable)    status. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


FILE  TRANSFER  -  Addresses  and  transfers  stored  text  or  data 
files  from  one  computer  storage  system  to  another,  or  from 
one  station  (assuming  local  storage)  to  another,  etc. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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ELECTRONIC    SPREAD    sheet   -   Allows    the    user    to    construct 
columns    and    rows    of    numbers.      Column   and    row    totals   are 
automatically   corrected    as   individual  members    in   the  matrix 
are  modified. 

Essential;   would  use   frequently 

Nice   to  have;   would  use  occasionally 

Do  not   need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


data  FILE  -  Allows  the  user  to  create,  edit,  maintain  and 
retrieve  data  in  a  format  (with  edit  criteria)  as  specified 
by  the  user. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 
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report  writer  -  Allows  a  user  to  subject  a  data  file  to 
elementary  numeric  manipulations  and  to  retrieve  the  results 
in  a  report  format  specified  by  the  user. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


INTERROGATE  -  Allows  a  user  to  display  a  record(s)  meeting 
specified  selection  criteria  from  a  data  file  specified  by 
the  user. 

Essential;  would  use  frequently 

Nice  to  have;  would  use  occasionally 

Do  not  need 

If  this  capability  is  made  available,  specify  which  office 
products  and  their  associated  tasks  would  be  accomplished 
more  efficiently. 

Products  Tasks 


19.  To  what  level  would  you  increase  your  usage  of  the  system  if 
it  offered  all  the  capabilities  described  above. 

0-5%    (1/2  hour) 

6-12%   (1  hour) 

13-25%  (2  hours) 

26-37%  (3  hours) 

38-50%  (4  hours) 

Over  50% 


20.  How  would  you  rate  the  reliability  of  the  hardware? 

Excellent    Good    Satisfactory   Poor 

21.  What  hardware  devices  have  required  the  most  maintenance? 

CRT 

Keyboard 

Printer 

Disc  Drives 

Telecom  Interface  or  Telephone  Lines 

Other  (Specify)  


22.  If  hardware  failure  occurs,  how  would  you  rate  the  response 
time  for  maintenance? 

Adequate     Slow    Too  Slow 

23.  Do  you  have  a  personal  contact  for  questions  and  problems 
concerning  OA  software  applications? 

No Yes  

In-house  

Vendor   

24.  How  often  have  you  experienced  software  problems? 

Rarely  (Once  every  3  months)   Often  (Twice  a  month) 

With  what  capability: 
Occasionally  (Once  a  month)       

25.  How  would  you  rate  the  response  time  for  software 
maintenance? 

Adequate     Slow     Too  Slow 
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26.  How  often  does  a  system  failure  or  crash  occur? 

Rarely  Occasionally      Often 

(Once  every  3  months)    (Once  a  month)        (Twice  a  month) 

27.  Are  there  backup  and  recovery  procedures  in  the  event  of 
system  failures? 

No  Yes  

28.  Do  you  have  to  repeat  (or  re-enter)  work  already  done  after  a 
system  failure  or  crash? 

No  Yes 

29.  Does  the  potential  unreliability  of  the  system  inhibit  you 
from  using  it  for  quick  response  or  fast  turn-around? 

No  Yes  

(Please   comment) 


30.    Are    you    given    satisfactory    notification    of    scheduled 
downtime? 


No  Yes 


31.  What  are  the  most  frequent  reasons  for  the  scheduled 
downtime? 

Scheduled  maintenance  

Telecommunications  


Software  enhancements_ 
Hardware  modifications. 
Other  (specify) 


32.  Do  you  or  would  you  like  to  use  your  workstation  for  non-OA 
functions? 


No Yes. 


Programming 
Data  retrieval 
Other  (Specify) 


33.  Do  you  by-pass  the  OA  functions  and  use  your  workstation  to 
directly  interact  with  system  software? 

No  Yes  (If  yes,  why) 


34.  To  what  level  would  you  increase  your  cumulative  usage  of  the 
system  if  your  workstation  accumulatively  added  these 
features: 

USAGE 

06-12%  13-25%  26-37%  35-50%  51-62%  63-75% 
(1  hr)  (2  hr)  (3  hr)  (4  hr)  (5  hr)  (6  hr) 

WORKSTATION 
WITH: 

Intelligence    

Local  Storage   

Local  Plotter   

Engin.  Graphics  

Voice  

488  Interface   


35.  What  do  you  like  the  most  about  your  OA  system? 


36.  What  do  you  like  the  least  about  your  OA  system? 


Data  Collection  Monitor: 


Received  . 
Reviewed  . 
Tabulated. 


/8/. 
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SUMMARY  OF  SURVEY  QUESTIONS  AND  RESPONSES 


This  is  a  summary  of  the  questions  contained  in  the  NALTOACS 
survey  listed  on  pages  1  through  20.  This  summary  examines  each  survey 
question  to  determine  whether  the  responses  in  each  question  lists  a 
response  judged  positive  (by  the  author)  for  OA  in  general  or 
negative/neutral  for  OA  in  general. 
Question 

negative/neutra 1 


1  -  9: 

10 


11 

12  -  16 

17 

18 


Series  of  13  OA  systems  functions  the  respondent 
is  asked  to  judge.  Each  of  the  functions  has  six 
response  areas;  five  of  these  areas  list  positive 
responses  for  OA  first.  Therefore,  there  are  65 
positive  and  13  negative/neutral 

positive 

negative/neutra 1 

positive 

Series  of  17  additional  OA  functional 
capabilities  that  are  available.  Each  function 
has  one  positive  response.  Therefore,  there  are 
17  positive  responses. 


19: 

negative/neutra 1 

20: 

positive 

21: 

negat i ve/ne  utr a 1 

22: 

positive 

23: 

negative/neutra 1 

24  • 

-  26: 

positive 

27: 

negat i ve/ne  utr a 1 

28  ■ 

-  29: 

positive 

67 


30  -  34:     negative/neutral 
35:     positive 
3  6 :     negat ive/ne utra 1 
The  NALTOACS  questionnaire  provides  the  opportunity  for  124 
responses.  Of  that  124,  92  of  the  responses  are  judged  by  the  author 
to  be  positive  ( in  favor  of  OA  in  general )  or  the  first  response 
option  listed  is  positive.  The  remaining  32  are  considered  negative  or 
neutral  of  OA  in  general  or  the  first  response  option  listed  is 
negative/neutral.  Dividing  the  number  of  positive  responses  (92)  by 
the  total  number  of  responses  (124)  yields  a  figure  of  74%. 
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APPENDIX  B 

PROJECTIONS  OF  POTENTIAL  SAVINGS  THROUGH  THE 

INCORPORATION  OF  OFFICE  AUTOMATION  EQUIPMENT 

AT  NAVAL  WEAPONS  CENTER  CHINA  LAKE,  CALIFORNIA 

Table  1,  Table  2,  and  Table  3  are  graphic  illustrations  of 

estimates  made  by  the  Maddox  Group  to  illustrate  the  opportunities  to 

automate  various  office  functions  at  NWC  China  Lake  (Maddox,  February 

1981). 

Pages  73  through  77  of  this  Appendix  are  taken  from  the  ATI PS  Concept 

Development  Paper  (NWC,  March  1982). 
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I.  Personnel  Costs.  Direct  costs  are  the  salaries  of  personnel 
required  to  operate  the  ATIPS.  The  salary  schedule  for  a  technician  is  averaging 
$25,951  annually.  For  the  purposes  of  this  economic  analysis,  the  figure  of  $25,951 
was  inflated  by  9  percent  to  account  for  cost  of  living  increases  between  FY81  and 
FY82,  adjusted  by  10  percent  to  account  for  cost  of  living  increases  between  FY82 
and  FY83,  then  adjusted  by  31  percent  to  conform  to  the  guidelines  set  forth  by  the 
A-76  Cost  Comparison  standards. 

Average  salary  for  FY81  =  $25.9  K 
Average  salary  for  FY82  =  $28.3  K 
Average  salary  for  FY83  =  $31.1  K 
Adjustment  prescribed  by  A-76   31  percent 

Total  adjusted  average 

technician  salary  =       $40.7  K 

The  adjusted  salary  of  $40.7  K  is  used  as  the  salary  of  the  ten 
personnel  who  are  required  for  the  operation  of  ATIPS.  This  staff  will  be  used  to 
operate  and  maintain  the  system,  perform  restart  and  recovery,  and  if  time 
permits,  training  and  helping  users  become  familiar  with  ATIPS. 

2.  Maintenance  Costs.  Maintenance  costs  will  not  be  incurred  in 
FY83,  since  it  is  assumed  that  the  first  increment  of  the  ATIPS  will  not  be 
functional  until  FY84.  Monies  will  be  committed  during  the  fiscal  year  requested, 
but  delivery  of  hardware  will  be  staged  over  time  to  facilitate  installation  and 
training.  Annual  maintenance  costs  are  predicated  on  estimated  maintenance 
contract  costs  of  $250  per  terminal,  $5,000  per  processing  unit,  and  $10,000  per 
shared  resource.  The  first  increment  of  ATIPS  (FY84)  will  require  60  terminals, 
three  processors,  and  one  OCR  for  a  maintenance  cost  of  $40,000.  Costs  in  FY85 
reflect  the  use  of  an  additional  480  terminals,  10  processors,  one  OCR,  and  one 
printer  for  an  annual  cost  of  $230,000.  Costs  for  FY86  include  the  third  increment 
of  540  terminals,  11  processors,  one  OCR,  and  one  printer  at  an  annual 
maintenance  cost  of  $440,000.  Costs  for  FY87  include  the  fourth  and  final 
increment  of  540  terminals,  11  processors,  one  OCR,  one  quality  printer,  and  one 
video  disc  at  an  annual  maintenance  cost  of  $660,000. 

3.  Other  Costs.  Other  costs  include  the  annual  costs  for  supplies 
(paper,  ribbons,  printwheels,  etc.),  equipment  and  building  rental,  if  applicable,  and 
for  utilities  (electric,  heat,  etc). 

4.2.5.2    ATIPS  Benefits.    Benefits  are  both  quantitative  in  dollar  savings  and 
qualitative  in  terms  of  a  more  timely  higher  quality  product. 

a.        Quantitative  Benefits.  Quantitive     benefits     are     derived     from 

calculating  dollar  savings  attributed  to  the  more  efficient  operation  provided  by 
ATIPS,  resultant  cost  avoidance  savings  and  reduced  expenditures  of  the  current 
system. 

1.  The  results  of  a  requirements  analysis  performed  by  The  MATTOX 
Group  in  December  1980  through  February  1981  at  NWC,  shows  that  ATIPS  will 
result  in  a  20  percent  savings  in  time.  This  savings  will  result  in  additional  time 
being  made  available  to  a  staff  member  to  work  on  projects  and  is  converted  to 
cost   savings   by    multiplying   the    20    percent    factor    against    the    persons   salary 
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(including  all  benefits).  This  value  agrees  with  information  from  other  similar 
office  automation  projects  from  both  Government  and  industry  and  is  felt  to  be 
conservative.  Increases  in  efficiencies  far  above  this  20  percent  value  have  been 
reported  in  well  organized  and  scheduled  office  automation  activities.  Industry 
publications  indicate  typical  productivity  increases  of  from  10  percent  to  30 
percent  using  available  systems.  Amoco  Production  Research  Co.  claims  a  60 
percent  increase  in  the  productivity  of  its  personnel.  Booz,  Allen,  and  Hamilton 
estimates  a  20  percent  increase  in  productive  time  with  the  adoption  of  new  office 
automation  technology  with  the  greatest  payoff  coming  to  senior  managers  in  the 
form  of  enhanced  decisionmaking.  For  the  purpose  of  calculating  cost  avoidance 
based  on  the  20  percent  savings,  an  average  weighted  salary  (average  of  all  salary 
classes  including  benefits  in  FY83  dollars)  of  $43.8  K  is  used. 

2.  This  value  is  obtained  by  proportionally  averaging  the  number  of 
members  in  each  Job  category  times  their  salaries.  These  salaries  for  each  job 
category  are:  Scientist  and  Engineer  ($35,241),  Technicians  ($25,950),  Clerical 
($14,077),  and  Administrative  ($26,634).  The  resultant  average  of  $27,876  is  then 
inflated  by  9  percent  to  account  for  cost  of  living  increases  between  FY81  and 
FY82,  and  by  10  percent  for  FY82-FY83  increases,  and  adjusted  by  31  percent  to 
conform  to  the  guidelines  set  forth  by  the  A-76  Cost  Comparison  Standards  for 
benefits. 

Average  salary  for  FY81  =  $27.9  K 
Average  salary  for  FY82  =  $30.4  K 
Average  salary  for  FY83  3  $33.4  K 
Adjustment  prescribed  by  A-76  31  percent 

Total  adjusted  average 

salary  =  $43.8  K 

3.  In  FY85,  it  is  estimated  that  650  employees  will  potentially  be 
affected  by  ATIPS  from  bringing  on  line  540  terminals  and  will  realize  an  increase 
in  efficiency  of  20  percent  at  a  cost  avoidance  savings  of  $5.7  M.  This  estimate  of 
the  relationship  of  employees  to  terminals  is  based  on  the  known  synergistic 
effects  that  come  about  through  social  interaction  between  employees  (e.g., 
roommates  sharing  a  terminal  exchange  information  to  the  gain  of  each).  In  FY86 
an  additional  950  personnel  will  potentially  be  affected  by  ATIPS  from  adding  an 
additional  540  terminals.  This  results  in  an  additional  savings  of  $8.3  M  or  a 
cumulative  annual  cost  avoidance  savings  of  $14  M.  In  FY87  an  additional  540 
terminals  will  be  added  providing  use  to  an  additional  900  personnel.  This  results  in 
an  additional  savings  of  $7.9  M  or  a  cumulative  annual  cost  avoidance  savings  of 
$21.9  M.  This  annual  savings  will  be  the  same  for  the  remaining  5  years  since  no 
further  terminals  are  anticipated  after  FY87  as  a  part  of  this  purchase.  Labor 
savings  due  to  cost  avoidance  over  the  first  8  years  of  ATIPS  in  constant  FY83 
dollars  is  $151  M.  These  savings  are  broken  out  for  each  year  over  the  system's  life 
cycle  and  are  shov/n  in  Figure  4-7. 
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4.  Other  dollar  savings  can  be  achieved  through  a  reduction  in  the 
travel  budget  made  possible  by  the  enhanced  communications  of  ATIPS,  reduction 
in  the  current  maintenance  costs  of  the  status  quo  system,  and  reductions  in  costs 
for  the  Center  distribution  system  (vehicle  maintenance  and  operations  costs).  A 
breakout  of  these  potential  cost  savings  in  FY83  dollars  shown  below  are  as 
follows: 

COST  SAVINGS  ELEMENT  LIFE-CYCLE  COST  SAVINGS 

1.  Off  Center  Travel  $8,820  K 

2.  Maintenance  Costs  of 

Office  Equipment  $4,200  K 

3.  Off  Center  Travel  Time  $1,470  K 

5.  The  above  savings  in  travel  are  attributed  to  an  estimated  IS 
percent  reduction  in  traveling  requirements  made  possible  by  the  enhanced 
communications  of  ATIPS,  allowing  S<5cE's  to  communicate  with  off-Center 
personnel,  data  bases,  and  libraries,  etc.  NWC  averages  7,000  trips  per  year  in  off- 
Center  travel  at  an  average  cost  per  trip  of  $1,200.  The  study  estimates  that 
approximately  15  percent  of  this  travel  can  be  saved  each  year  through  the  use  of 
ATIPS.  This  amounts  to  $420  K  the  first  year  ATIPS  is  fully  operational.  Using 
FY83  dollars,  the  life-cycle  (8  years)  cost  savings  realized  amount  to  $8,820  K. 

6.  It  is  expected  that  savings  will  result  from  a  reduction  of  time 
spent  in  a  travel  status.  Assuming  one  day  of  lost  time  for  each  of  the  trips 
avoided  above  the  savings  the  first  year  would  be  $70,000  and  $1,470,000  for  the  8 

years. 

7.  Savings  of  the  current  maintenance  results  from  decreased  wear 
through  less  usage  on  such  items  as  copiers,  typewriters,  and  other  office 
equipment.  During  FY  80,  for  example,  NWC  spent  $260  K  to  maintain  office 
equipment,  $20  K  per  month  on  copy  machine  paper  and  $600  K  on  copy  machine 
rentals.  In  the  same  period,  $110  K  v/orth  of  new  word  processing  equipment  was 
purchased  and  $139  K  was  spent  to  lease  v/ord  processing  equipment.  These  figures 
were  converted  to  FY83  dollars  and  it  is  assumed  that  50  percent  of  the  related 
cost  can  be  avoided. 

8.  Additionally,  it  is  expected  that  there  will  be  a  cost  avoidance  of 
$322  K  (FY83  dollars)  in  FY84  in  that  the  Center  will  defer  anticipated  purchases 
of  office  supplies  and  equipment  to  wait  for  ATIPS  to  supply  the  necessary 
capability.  The  savings  due  to  cost  avoidance  over  the  expected  8-year  life  cycle 
of  the  ATIPS  project  total  is  $166  M,  and  is  plotted  by  installation  phase  in 
Figure  4-8.  This  graph  shows  the  acquisition  costs  and  the  total  cost  savings  as  the 
equipment  is  installed  and  becomes  operational.  A  very  early  break  even  point  (3 
months)  is  shown.  However,  a  payback  period  of  1.25  years  (the  inflection  point  in 
the  cost  savings  line)  is  claimed  based  on  the  sensitivity  of  the  estimated  20 
percent  productivity  index  rate. 
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